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Executive Summary 


Education systems in under-resourced environments face several challenges, some of 
them exacerbated by the COVID-19 pandemic. A possible way to address some of the 
challenges is to apply information and communication technologies. However, effectively 
integrating technology into education systems is a complex task. In this paper, factors 
contributing to successful integration of technology in education are explored, with a focus 
on under-resourced contexts. Case studies of successful technology implementation in 
education systems are discussed and analyzed to identify the factors that drive success. 
The analysis is framed using the reform strategy offered by The World Development Report 
2018 (Learning to Realize Education’s Promise). Thisis expected to provide policymakers 
and practitioners a way to align their education technology initiatives and strategies with 
the larger education reform agenda. Key lessons identified from the analysis are as follows. 
First, it is necessary to articulate what precisely does the technology intervention 
change/enable. Second, it is important to better understand the context to develop 
technologies and implementation strategies that fit the operating context. Third, it is 
essential to regularly monitor and evaluate programs and to feed that information into 
continuously improving design and implementation. Fourth, through the entire cycle of 
technology implementation, stakeholders must be consulted, understood, and 
empowered. However small the intervention, realizing the potential of technology tools in 
education requires keepingin mind the big picture offered by these lessons. 


1. Introduction 


Investments to introduce various forms of information and communication technologies 
(ICTs) in public school education systems around the world rapidly increased over the last 
two decades and witnessed significant expansion during the COVID-19 pandemic. The 
classification and magnitude of such investments vary based on what report one chooses 
to read. Nonetheless, most of these investments have attempted to reproduce hardware- 
centric educational technology projects that were conceived in, and hence better suited to, 
resource-rich environments. The evidence on their effectiveness is either limited or largely 
mixed. What will it take to succeed, then, when thinking about ICTs in education for the 


long run? 


2. Technology for what? 


“If | were given one hour to save the planet, | would spend 59 
minutes defining the problem and one minute resolving it.” 
- Anonymous 


Challenging the status quo of school systems is hard. Ideas to re-imagine the system, many 
involving the use of ICTs, have been proposed over decades. In 1913, Thomas Edison, an 
American inventor, predicted that books would become obsolete in schools and be 
replaced by motion pictures. Around the same time, Edward Thorndike, a Columbia 
University educational psychology professor, imagined ‘adaptive textbooks’ that would 
change based on individual students’ achievements. Sidney Pressey, an Ohio State 
University professor in 1933, and later B.F. Skinner, a Harvard University psychology 
professor in the 1950s, posited and even modeled teaching machines. In 1972, computer 
scientist Alan Kay conceived the Dynabook, “A Personal Computer for Children of All Ages” 
as yet another idea of personalized learning using what we today know as tablet 
computers. A lot of these ideas came from the Western world, especially the United States, 
and constitute the rich and fascinating history of envisioning ICT use in education systems. 


Itis pertinent, therefore, to highlight that ‘technology innovations’ have been a key factor in 
shaping education systems around the world. The fundamental approach to solving real- 


world problems using technology requires sufficient rigor in defining the problem from an 
end-user perspective, articulating why it’s important, developing ideas and re-imagined 
processes that may be enabled or expanded/scaled with the use of technology tools, and 
ensuring that the end-users are part of the solutioning exercise. The ideas of Edison, 
Pressey, and Skinner highlighted in the previous paragraph perhaps reflected increasing 
demand for education as the problem and ‘mechanization’ as a way of improving efficiency 
in an ever-expanding classroom. The ideas of Thorndike and Kay probably identified how- 
to-teach-at-the-right-level and how to encourage individual exploration as the problem and 
offered ‘personalization tools’ as a solution. 


Before envisioning an ICT-enabled model for addressing the problems facing school 
education today, it is worthwhile pausing and reflecting on what we are solving for in 
today's regional context. Several regions around the world are faced with challenges of 
access, quality, andinclusion in education. In response, the World Development Report 
(WDR) 2018 proposed a reform agenda with three complimentary strategies to realize 
education's promise. This paper attempts to articulate key problems that need to be solved 
in executing the three strategic areas of the WDR reform agenda, and how ICT can address 
some of those. 


3. Technology as a potent tool to address 
challenges in under-resourced environments 


The World Development Report 2018's proposed reform strategy involves the following: 
e assessing learning to understand the current state of the system as well as set 
direction for the reform agenda, 
e acting on evidence to resolve barriers to learning, and 
e aligning all actors to make the whole system work for learning. 


3.1. Assessing Learning 


In the past decade and a half, assessments have been central to the discourse on 
education reform around the world and have been responsible for bringing into public 


consciousness the issue of low levels of learning in most national education systems. A 


2012 World Bank report? highlighted three main drivers of the quality of information 
produced by an assessment system: the enabling context, system alignment, and 
assessment quality. Broadly speaking, the enabling context covers policies, resources, and 
implementation arrangements for assessment activities; system alignment covers the 
connection between assessments and other components of the education system such as 
learning goals, standards, curriculum, and teacher training; and assessment quality covers 
the psychometric quality of the instruments, processes, and procedures used for 
assessment. Analyzing a country’s assessment systems using such criteria offers interesting 
insights into specific problems/areas that need to be addressed/strengthened and if or how 
ICT tools might help. For instance, several countries grapple with issues of widespread 
cheating in high-stakes examinations. In other cases, there are issues with the alignment of 
national large-scale assessments to the constructs/content they were intended to measure, 
i.e., the curriculum or learning standards. Test-items/questions and procedures may also 
vary in quality across regions and years, making the results difficult to compare. The 
process of administering assessments and collecting and collating data may also be prone 
to errors at various stages, resulting in unreliable data, time-consuming analysis, anda 
slow and ineffective assessment-to-action loop. 


Some of the efforts to improve assessment systems have used ICT tools to improve the 
fidelity of the data, including integration of portable devices like mobile phones and tablets 
into data collection procedures, and using downstream analytical tools to shorten the 
assessment-to-action cycle. One example of interventions that have targeted system-wide 
changes is the ‘big-bang’ assessment reformin the country of Georgia in 2010, wherein all 
school graduation examinations moved to computer-adaptive testing (CAT), a type of 
testing that adapts to the ability level of the person taking the test.* There are several 
potential benefits of such a shift, ranging from a reduction in cheating (because test takers 
are shown different questions) and availability of more precise ability-estimates of 
candidates, to development of extensive item banks well-aligned with national standards 
that can lead to more reliable and comparable results over years. An evaluation by the 
Georgian Ministry of Education and Science concluded that the computer-adaptive testing 
in Georgia was more cost-effective compared to administration of the same tests using 
paper and pencil techniques, and that its introduction had met the goals of launching an 
efficient, fair, and objective approach to student assessment. 


3 (Clarke, 2012) 
4 (Bakker, 2014) 


The success of this model in Georgia, however, owes much to a number of enabling factors 
specific to the ‘context’. The small size of Georgia, and the presence of a well-developed 
internet infrastructure in its schools served as key ingredients to thinkin the direction of 
technologically advanced assessment delivery modes. A strong commitment from the 
government to introduce national tests, fund the recurrent costs, and guarantee continuity 
of operations underpinned the enabling context. The most important success factor, 
however, was the leadership of a highly competent and experienced testing institution 
(National Assessment and Examinations Center, NAEQ), and its ability to bring schools on- 
board. NAEC developed the CAT tool using its existing staff, with some training by 
international consultants in areas such as item calibration and test analysis using software 
packages. NAEC then conducted extensive outreach campaigns - led by its Director, a well- 
respected and trusted public figure - by visiting all regions and meeting with 
representatives of virtually all schools involved. The meetings focused on sharing 
information about registration and score dissemination processes, demonstrating working 
prototypes of the test delivery system and test items, and, most importantly, convincing 
schools that the tests would not be used againstthem and that the outcomes would not be 
disastrous as many feared. This was complemented by visits of technology specialists to 
each school for inspection of computers and connectivity. This spurt of activity was 
followed up with sharing of information about the test with the publicand media, setting 
up of a dedicated Frequently Asked Questions section on the website and social media 
channels, an open-door policy of interaction with the media, and launching of a web-based 
practice test. Itis worth stressing that while decisions around the ‘right technology’ were 
important, overall success was rooted ina much more comprehensive analysis of needs 
and goals, and a rigorous implementation and stakeholder consultation process. In fact, 
the role of the NAEC in successful implementation cannot be overstated; although, it was 
politically not responsible for the decision of administering computer -adaptive testing, it 
took on the onus of engaging the ecosystem through outreach, ensuring stakeholder buy- 
in and successful implementation. 


Another example of efforts to improve national assessment systems is Afghanistan's 
Monitoring Standards in Educational Growth (MTEG) program, launched in 2013, which is 
designed to assess the learning progression and trends in achievement outcomes of 


students at key stages of schooling (Classes 3, 6 and 9).° The program has developed a 
computer-based assessment for its Class 3 assessments®. 


3.2. Resolving Barriers to Learning 


While assessments tell where one currently is and how far one has moved, this information 
is only helpful when acted upon. Drawing from an ever-growing body of evidenceon what 
works, the World Development Report 2018 highlighted four key school-level ingredients 
for learning: prepared learners, effective teaching, learning focused inputs, and the skilled 
management and governance that pulls them all together. 


Cae B Prepared Learners 


The World Bank has laid out a three-pronged strategy to accelerate progress towards the 
goal of ensuring all children have the opportunity to reach their full potential ’: establishing 
an enabling environment; ensuring that the portfolio of programs covers healthcare, 
nutrition, child protection, social protection and early learning; and, monitoring and 
assuring quality through compliance with quality standards and data availability. Similar to 
the case of assessments in the previous section, using such a framework in analyzing a 
country’s Early Childhood Development (ECD) system ensures the primacy of the needs 
and aspirations of all stakeholders in designing and delivering solutions, surfaces gaps, and 
thereby, generates insights on whether and ways in which ICT can play a role in solving the 
identified challenges. 


For example, wide range of Government departments delivering similar services due toa 
lack of inter-sectoral coordination mechanisms for designing, delivering and monitoring 
services and outcomes may become constraints to an enabling environment. Lack of 
quality assurance mechanisms may lead to proliferation of poor-quality curriculum and 
learning resources, untrained professionals, and ill-equipped child-care centres that 
adversely affect children’s growth and learning. 


° (ACER Centre for Global Education Monitoring, 2016) 
® See Box 1 for a detailed description 
7 (Neuman & Devercelli, 2013) 


Use of technology can sometimes provide governments simpler ways to deliver quality ECD 
interventions at scale. For example, one of the most inexpensive and widely available 
technologies, the radio, has been used to deliver interactive radio instruction (IRI) targeting 
the most vulnerable children in remote communities, with very encouraging results. IRI is a 
distance education system that combines radio broadcasts with active learning to improve 
educational quality and teaching practices. The first phase of Radio Instruction to Strengthen 
Education (RISE) project in Zanzibar used IRI to deliver songs, stories and physical activities 
to a group of underserved young children (preschool to Grade 2) and a lightly-trained adult 
facilitator in Zanzibar's two lowest-performing districts. The project was led by Zanzibar’s 
Ministry of Education and Vocational Training (MOEVT). MoEVT started with empowering 
educators through capacity building of its educators in developing interactive audio 
programs and accompanying instructional and learning materials, producing audio and 
video programs, and conducting monitoring and evaluation activities. These educators 
could then integrate math, life skills, and English and Kiswahili literacy lessons into 
comprehensive 30-minute programs. The programs complemented the standard 
curriculum, engaged listeners through locally based stories and activities, and could 
support teachers with minimal ECD instructional experience. Early childhood learning 
centers (TuTu Centers) were then established in areas with poor access to ECD and 
education, and RISE staff trained local community members as mentors to guide learners 
through the TuTu broadcasts and post-broadcast activities in non-formal settings identified 
by the community. MoEVT also trained its district officials in IRI implementation, in addition 
to monitoring and evaluation. The programs were broadcast on the government- 
sponsored radio station as well as through portable media player technology. The initiative 
delivered deep impact, exhibited through significant test score gains across all three tested 
subjects (Kiswahili, Math and English)®?. 


The success of the project once again highlights that the ‘right technology is the one that 
suits the context; inthis case the one that canreach vulnerable children in remote areas 
and is easy to use. It also brings to fore another critical element for successful technology- 
driven interventions: robust change management and empowerment of the last-mile 
actors. Capacity building at various levels and engagement with the communities not only 
enabled alignment for easy delivery of the project, but also aided the MoEVT’s ability to 
take over implementation and institutionalize it in just a few years. 


8 (World Bank Group, 2017) 
° (EDC, 2015) 
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Other noteworthy models of technology use in ECD are Storyweaver (India) and Ready4kK! 
(USA). Storyweaver (India) is an online platform that gives children and adult-caregivers 
opportunities to read, print and even discuss levelled stories written in multiple languages, 
and also allows illustrators, authors and translators to collaborate to create and translate 
stories. 


Ready4k!"° an initiative that was implemented in the San Francisco Unified School District 
during 2013-14 and 2015-16, aimed to address the large variations in learning experiences 
for young children with varying family backgrounds. One study had found that by the age 
of four, children in “welfare families” hear about3 million fewer words than children in 
“professional families”. Ready4k! intended to bridge this gap by supporting parents of 
young children to provide a rich home-based learning experience. However, most existing 
parenting based interventions suffered from challenges of low access, high cost and parent 
dropouts, and therefore, had shown very little success. Given this context, Ready4K! arrived 
at a smart design through a careful consideration of the context and factors that affect 
implementation of parental support programs. Firstly, it employed a widely available and 
inexpensive technology, i.e. of text messages, as the primary mode of delivery of the 
intervention. This ensured thatthe project reached a// communities, including the under- 
resourced Black and Hispanic communities which had the highest drop-out rates from 
parenting programs. Secondly, Ready4k! broke down the complexity of parental support 
into an accessible package of three messages a week - one message with a simple bite- 
sized step for the parents to follow, along with two messages for encouragement and 
reinforcement. The program resulted in increased parental involvement at home and 
school, leading to gainsin early literacy. This initiative exhibits the promise of small bits of 
information, and widely available, low-cost and scalable technology solutions in changing 
entrenched and complex adult behaviors, echoing the importance of understanding the 
context and identifying the right technology for that context. 


Similar to other challenges in the education systems, technology may not be the answer to 
the challenges of ECD. However, some right approaches that can complement and enable 


activities related to ECD are possible, provided a clear understanding of the context in 
which ICT tools can help. 


10 (York, Loeb, & Doss, 2017) 
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2.2. Learning Focused Inputs and Effective Teaching 


While a major part of the strategy to prepare learners entails focus on early childhood care 
and education, the strategy includes directing attention to the value of increasing 
motivation and effortin school and the need for increased and improved opportunities to 
learn at home. There is a plethora of solutions that seek to provide rich and engaging 
lessons, activities, resources and support for both the classroom and home, mostly on 
literacy and numeracy curricula for primary grades and above. They range from tools that 
aid reading at home to those that enable sharper learning experiences inschool. Some of 
these tools may be more resource-intensive than others, but there are many that make use 
of existing and easily accessible technologies. 


Available in India and Pakistan, Worldreader is a widely-available tool that allows children 
and adults to read stories from mobile devices, thereby increasing the exposure to quality 
content and support outside the classroom. There have been pilots in Pakistan that use 
SMS technology to deploy short daily assessments. Solutions using SMS and Interactive 
Radio Instruction (IRI) demonstrate the potential of well-designed technology-based tools 
to deliver improved learning experiences bothin and out of classrooms, particularly in low- 
cost and limited-mobility environments. Another example is that of Mindspark, a 
mathematics learning program that uses adaptive-learning technology. A study in India"' 
found that students who used Mindspark after-school for 4.5 months, showed gains that 
were greater than those observed in almost any evaluation of education interventions in 
poor countries. 


Technology can also be used to supportteachers in improving their instruction. In Uganda, 
UNICEF's MobiStation - referred to as a digital ‘school in a box’ - is equipped to facilitate a 
richer learning experiencein both formal classroom and informal communal settings. It 
does so through a combination of laptop, speaker, and projector rolled into one portable 
suitcase. While hardware-assisted systems and software preloaded with content constitute 
most technology-enabled tools, there are a few examples that go beyond to set up virtual 
or physical platforms that support content creation, dissemination and consumption. The 
Digital Infrastructure for School Education (DIKSHA) platform in India is one of such 
platforms. It is an open learning platform with a collection of learning resources in literacy 


™ (Muralidharan, Singh, & Ganimian, 2016) 


12 


and numeracy. The platform allows for creation and publishing of interactive teaching 
content for learners. The created stories, assessments, worksheets, simulations, games 
and activities can be accessed through a website and mobile app. All the education content 
developed on it is linked to a particular curriculum, and can be used bystudents, parents, 
schools and tutors. The platform provides analytics and dashboards that help track ratings, 
engagement and learning levels of users. With availability in 12 languages and enablement 
of content creation by teachers, parents and content creators, it represents a step towards 
democratization of content creation and rating. 


ICTs and properly developed multimedia materials can also enhance the initial preparation 
of teachers by providing good training materials, facilitating simulations, capturing and 
analyzing practice-teaching, bringing world experience into the training institution, and 
training potential teachers in the use of technologies for teaching/learning. For example, 
teachers in the Indian state of Karnataka convene under the umbrella of the Subject 
Teacher Forum (STF) to engage in discussions related to their subject on online 
communities and create and share open digital educational content. All of these are tools 
that were built for the context, address both content and infrastructure issues, use locally 
aligned or created content, and have a component of training teachers in using them well. 


Cae | Skilled Management and Governance 


At the school/institution level, technologies are crucial in such areas as admissions, student 
tracking, personnel management, staff development, and facilities management. At the 
system-wide level, technologies provide critical support in domains such as school 
mapping, automated personnel and payroll systems, management information systems, 
communications, and information gathering, analysis, and use. 


ICT can be valuable in storing and analyzing data on education indicators, student 
assessment, educational infrastructure (physical and human), cost, and finance. More 
importantly, they can assist in constructing and assessing policy scenarios around different 
intended policy options to determine requirements and consequences and to help select 
those that are the most appropriate. An example of use of ICT for management and 
governance may be found in the Indian state of Karnataka. It is an example that is equally 
important for the manner of its implementation, as it is for its scale and sustainability. 
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Karnataka has over 60,000 schools (almost three-fourth of them managed by the 
government) and more than 450,000 teachers (with more than 50 percent of them 
employed in government schools). Through the 1950s to early-1990s, Karnataka’s teacher 
recruitment and transfer processes were largely decentralized, ad-hocand dominated by 
patronage networks. District and state level bodies, comprising local members of the 
legislative assembly and other influential political leaders, conducted the recruitment of 
primary and secondary school teachers. Teacher transfers were also largely undertaken as 
a means of punishing errant teachers and rewarding those with connections. Susceptible 
to corruption, the existing systems and practices invited widespread criticism and 
dissatisfaction from teachers'2. Given this context, the Government of Karnataka made a 
series of key reforms in improving the teacher recruitment and transfer systems. Amongst 
all the changes made, centralization of teacher recruitment and transfer processes within 
the office of the Commissioner of Public Instruction (CPI) in Bangalore and laying out of 
transparent criteria for teacher selection and transfer were the two most important ones. 
Once the problem was well defined and solutions envisioned, technology played a crucial 
role in the actual implementation. The sheer scale of the implementation entailed a large 
volume of administrative tasks, and this was facilitated through an iterative installation of 
technology-enabled systems and processes, often adapted from other sectors of education 
where they had already met with some success. As these rules and processes were 
continuously improved and several new acts were enacted in recognition of success of 
executive orders, the technology systems were also revamped, ensuring sustained use over 
time. Thus, clarity in problem and solution definition, evidence-based and iterative manner 
of implementation, and support from leadership were key to the success of using 
technology in reforming teacher recruitment and transfers at scale and in making the 
processes andsystems more transparent, objective and fair. 


Similar to the system in Karnataka, the Sindh School Monitoring System (SSMS) in Pakistan 
is also an example of an expansive system-wide technology system that was created to 
improve governance, accountability and service delivery. It constitutes field based monitors 
who use smart phones and biometric devices to monitor schools ona regular basis on a 
variety of attributes such as school status, infrastructure, facilities, staff and student 
attendance, which are then transmitted in real-time to centralized dashboards in the 
Education and Literacy Departments and Directorate of Monitoring & Evaluation. This helps 


planning and informs decision-making by the Sindh Government through review meetings 


12 (Ramachandran, et al., 2018) 
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at the districtlevel. With an iterative design that has resultedin the system spanning 29 
districts, monitoring of more than 26,200 schools, and biometric verification of 210,000 
education staff, this system has helped the Government identify and take action against 
40,000 absentees and 6,000 absconders. This project, too, benefited from a detailed 
analysis of the existing condition of the public education system in Sindh, along with an 
enabling environment that was provided by the reform agenda of Sindh Government's 
education sector plan, rather than being a standalone initiative. 


Box 1: Showcase Stories from South Asia 


Monitoring Trends in Educational Growth (MTEG), Afghanistan 


The Monitoring Trends in Educational Growth (MTEG) Program was launched in 2013 by 
the Learning Assessment Unit of the Afghanistan Ministry of Education, in partnership with 
the Centre for Global Monitoring at the Australian Council for Educational Research (ACER- 
GEM). Operational till 2017, MTEG was conceptualized with the purpose of providing policy 
makers with information about the state of education outcomes in Afghanistan. It aimed to 
doso by measuring trends in achievement outcomes in single classes over time (e.g. Grade 
3, year-on-year), and growth of achievement in cohorts throughout the school cycle 
(Grades 3, 6 & 9)'3. MTEG also provided information about the contextual factors around 
teaching, learning and home environments in order to provide a more complete picture of 


the influences on learning outcomes". 


Based on the results of the Grade 6 assessments that were conducted in 2013, the ACER 
team managing the program anticipated that students in Grade 3 would be spread across 
the spectrum of reading literacy, with significant numbers at very basic proficiency levels. 
To ensure accurate measurements for all children in Grade 3, including those with very 
basic levels of proficiency, the team arrived at a few innovations. 


Firstly, students were sorted into two groups, Emerging and Independent Readers, based on 
a short preliminary reading assessment that was conducted by the test facilitator. These 
two groups were directed to two different assessments, with the /ndependent Readers being 


13 (ACER Centre for Global Education Monitoring, 2016) 
"4 For further information about MTEG in Afghanistan see https://www.acer.org/genvkey- 
areas/system-strengthening/mteg. The Grade 3 MTEG report is forthcoming. 
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assigneda slightly more challenging test than the Emerging Readers. However, to accurately 
measure different levels even within these broad groups, both the assessments had items 
of arrange of difficulty levels. 


Secondly, the Grade 3 assessments were conducted on tablets that allowed for interactive 
input methods. The questions used touch-enabled responses such as Hot spot, multiple 
choice and drag-and-drop question types. The assessments also used integrated audio files 
for many of the questions, including for those assessing letter sounds, word recognition 
and listening comprehension. An important reason for students being read-aloud 
questions was that it allowed for more accurate testing of phonemic awareness, phonics, 
vocabulary, listening comprehension, and mathematical abilities of students who were not 
strong readers. Use of colourful interfaces, playful design and interactive input methods 
motivated students to participate in these assessments. Another very important advantage 
of conducting assessments on tablets was that a consistent assessment experience was 
provided to all students. The students also enjoyed the added support of accessing audio 
prompts as many times as they needed without the hesitation of requesting repeated 
clarifications from a human facilitator. Before the assessment commenced, each 
participant required around 15-20 min to acclimatize herself/himself with the software 
through guided walk-throughs of the interface and practice questions. The ACER team 
claimed that this resulted in almost all students being able to take the test without any 
challenges. The tablet-based assessment was available for offline delivery, meaning that no 
internet connection was necessary while students completed the assessment. 


Along with the benefits indicated above, the use of technology to deploy these 
assessments resulted in greater efficiencies. Data-collection on a tablet meant that the data 
was securely transferred to ACER's servers as soon as the tablets were connected to the 
internet, thereby reducing the fear of loss of data. Encryption and user passwords ensured 
that both assessments and test data would be secure even inthe event of loss of tablets. 


Technology-enabled implementation also resulted in an ability to complete assessments in 


a short period of time. Using this innovative approach to assessments had numerous 
benefits, including enhancing the efficiency and effectiveness of data collection in the 
MTEG program. This use of technology supports the collection of high quality and reliable 
data that can be used to inform policy makers about the learning outcomes of Grade 3 
students in Afghanistan. 
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Mindspark, India’? 


Educational Initiatives (El), the firm that designed the Mindspark software, had been 
founded on the idea that educational change could and would be catalyzed by 
demonstrating gaps in students’ learning. Consequently, El had been conducting 
assessments and educational research, and had assessed about 14 millionstudents since 
their founding in 2001. However, the slow pace of change in the education sector, despite 
El’s efforts in highlighting and explaining problems throughits assessments, along with 
having unraveled patterns in the misconceptions and challenges that children experience, 
inspired El to look beyond assessments. In 2008, the firm started developing Mindspark, a 
computer-assisted learning (CAL) software that enables students to learn mathematics and 
language at their own level and pace, and is meant to supplement classroom instruction. 


A mathematics-only program was launched in 2009, which is aligned to the requirements 
of a range of state and international curriculum, and currently packs content for Grades 1 
to 10. In 2016, an English reading and language program for Grades 4 to 9 was also 
introduced. The roll-out of the Mindspark program has been staggered over device -types, 
starting off with a desktop/laptop version in 2009, a tablet version in 2010, followed by the 
release of a device-agnostic version in 2013 that required onlyan internet browser. Finally, 
in 2014, El launched an offline version of Mindspark that would need only periodic internet 
connections, learning from the various cycles of its implementation. 


Consciously staying away from hardware innovations of digitized whiteboards and 
textbooks, the Mindspark system chose to aim for a cloud-based student-centric software, 
wherein the focus is on ‘absorption’ of learning rather than the simple delivery of 
curriculum. This is enabled through a multitude of features, such as the use of finely- 
graded questions, adaptive logic, and intelligent responses and modules. These ensure that 
students areon avery gradual learning trajectory, are directed to different questions based 
on whether their responses are correct or not, and are provided continuous feedback, 
remedial instruction and quizzes. To support students with a range of abilities in 
comprehension, it can provide voiceover support in mathematics, and has built-in graphics, 
animations and educational games for an extensive part of the content. Additionally, it 
implements a smart recognition system that not only rewards good performance, but also 


'° (Muralidharan, Singh, & Ganimian, 2016) 
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encourages desirable behaviors such as persistence in completing topics, spending time to 
read through feedback, and consistent use of the software. While these features drive 
student engagement with learning, the system also integrates a feedback system that 
allows students to rate each question (Like/ Dislike/ Excited/ Bored/ Confused) as well as 
provide typed feedback to El. 


In an illustration of good design, Mindspark implementation begins with a needs- 
assessment, infrastructure check, analyses, followed by dialogue among El, teachers and 
Principal to arrive at a decision on the suitability of Mindspark for the present challenges 
and circumstances. Only once Mindspark is found suitable, does the team begin 
implementation either ina few classrooms as a pilot or throughout the school, and provide 
teachers and students with orientation and training to use the software. Further, the 
program gives significant control to individual teachers, i.e. teachers can ‘activate’ content 
based on what they have taughtin the classroom, and then, inform their instruction using 
reports generated by the program. Over the years, the program has been tried and tested 
in a variety of contexts, starting from private schools to public schools in the Indian states 
of Madhya Pradesh, Gujarat and Rajasthan. In 2012, recognizing the need and potential for 
innovation in urban centres, the Mindspark program was implemented in after-school 
learning centres in the urban slums of Delhi. The implementation in Delhi has been 
rigorously tested by researchers associated with MIT's J-PAL. 


The study found that students who used the Mindspark program showed significant gains 


compared to the control group. In addition, the evaluation also found great relative gains in 
the levels of academically-weaker students compared to the control, since the rate of 
learningin the control group was minimal. While the study did not experimentally isolate 
the effects of the CAL system from the effects of group teaching and additional 
instructional time, it presents evidence that indicates that the most significant contribution 
was from the software. The software's abilities to instruct at the appropriate level and 
continually update the content for individual students based on their individual learning 
trajectories were found to be the primary drivers of success. 


On the matter of productivity, too, the research presents encouraging results. Estimates in 
the study suggest that, at a scale of 1,000 schools, (steady-state) implementation of the 
program would increase the government's annual per-child cost by less than 2%. In fact, 


even with high fixed costs, absence of economies of scale and low attendance, the program 
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delivered greater learning at lower costs than public spending. In the context of substantial 
percentages of childreninthe South Asia region severely lagging their levels, the study 
provides findings that could have remarkable implications. Using estimates, it hypothesizes 
that gains in learning levels by using the system regularly couldleadto students catching 
up with their appropriate grade-level standards. 


The study also presents some very compelling social and policy benefits of effectively 
implementing a system such as this. Complete personalization offered by a CAL system 
allows for children to learn at their own pace, which doesn't only mean that it can deliver 
content at a lower level and/or pace; it can also accelerate growth of the same students 
when they start learning at a faster pace. These adaptations come at no additional social 
costs to the children, such as changing their peers. It effectively ensures that students are 
not ‘tagged’ or fixed to one track, while allowing them to learn at their own pace. Secondly, 
refraining from commenting on the ‘ideal’ vision of how education systems should group 
children, a solution such as this allows nations to improve student learning without 
incurring heavy policy costs such as shifting away from the age-cohort based social 
grouping of students. In a way, it allows for governments to prioritize the right set of 
reforms at different stages of their own journey towards a transformational education 
system. 


Before concluding, a fine reading of the study report also unpacks some very relevant 
lessons for governments thinking about implementing such systems. While the CAL system 


was pivotal in delivering improved learning, instructors may have a very important role to 


play in the overall implementation. For example, in the study, which was implemented in 
an after-school centre in Delhi, the El instructor played a crucial role in encouraging regular 
student attendance, ensuring time on task, supervising school-homework completion and 
exam preparation during group-instruction period. Since some of these activities were 
demanded by parents, thinking through a holistic design of the intervention and obtaining 
buy-in from parents would be critical to successful implementation. In addition, the study 
also notes that the very nature of teaching-learning requires complex contextual- 
awareness and communication; thus, technology may serve to complement teachers, 
rather than substitute them. Secondly, parental demand for the Mindspark centres of the 
study was low in the absence of fee-waivers; thus, low-resource communities may not be 
able to access programs that charge a fee, and governments may need to think about 


delivering both hardware and software. 
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Tosummarize, the software's ability to instruct at the right level and pace, along with time- 


productivity considerations, and cost-effective and well-rounded implementation, make 


this a useful solution to test, implement, and iteratively improvein contexts with severe 
levels of inequity and resource constraints. 


3.3. Aligning Actors 


Reform components suchas assessing learning and acting on available evidence are not 
effective in and by themselves. Any reform initiative needs to gain traction and sustain 
within the systems itis implemented in. For this to successfully happen, itis important to 
identify and map different actors, and align and equip them with the necessary skills and 
orientation to direct all inputs towards those outcomes. However, aligning actors and 
hence, systems, to absorb a new set of reforms is a challenging task. There are primarily 
two ways in which alignment may fail to take place. Firstly, accountability fails when 
different actors hold the system accountable to different goals. For example, while 
students and parents may value learning, governments and school bodies may value only 
appropriate budgetary allocation. Secondly, while governments may task schools and 
education departments with the goal of learning, the funding, information and incentives in 
the system may not be aligned to learning. For example, while schools or education 
departments may very easily produce enrolment data, they won't have information 
systems Set up to produce such reliable learning data at high quality. Similarly, funding or 
retention of staff may not be linked to performance, butto other parameters, and thus, not 
incentivize learning. Thus, any reform initiative needs to be accompanied by an analysis 
that seeks to answer the following questions: 

e Areall stakeholders aligned with the shifts that the reform seeks to produce, and 

hence, aligned on the data that they will use to assess and improve the reform? 
e Is there adequate data on the outcomes that the reform seeks to achieve? 


Therefore, it is clear that the power of using crucial levers of change, i.e. goals, financing, 
and incentives, is predicated on the easy availability of high-quality information. 
Technology can play a vital rolein setting up the infrastructure in the form of Education 
Management Information Systems (EMIS) that can enable this. There are several 
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examples’ of successful building or consolidation of Education Management Information 
Systems (EMIS), including in Guatemala where data was made open to all through easily 
accessible website of the Ministry of Education; in Honduras where along with public 
access to data, relevant stakeholders were trained on using data; and Lithuania, where 
interest was generated among municipalities and school representatives, leading to the 
creation of local and school-level EMIS, and creation of an annual education report to 
measure student achievement. These are easily contrasted with examples that exhibited 
barriers suchas lack of clarity on the goals and purpose (Albania, Vietnam, India), lack of 
political will (Bulgaria, Latvia), not equipping key actors or involving them in design and 
dissemination of data (Bulgaria, Vietnam, Pakistan, Chad, Ghana), and lack of legal 
framework that supports the EMIS (Bolivia and Argentina). What these numerous examples 
illustrate is the criticality of the process of reform, of the efforts to align for accountability, 
even in the creation of systems for tightening the very drivers of accountability. 


One of the most critical design elements that enables a successful EMIS is 
reformers/practitioners supporting all relevant actors in identifying, articulating and 
prioritizing a problem, and then, helping create safe and conducive environments for 
iterative solutioning and innovation '’. An example of this design principleis exhibited by an 
organization called Open Learning Exchange (OLE, Nepal) that has dedicated strategies for 
partnering with the community, government, schools and other expert organizations. 
These strategies aim to build alignment on the vision, strategy and design of the solution, 
promise and ensure commitment to sustain a high level of performance, and hold all 
parties involved accountable to outcomes. Therefore, to set in motion and sustain the 
virtuous cycle of assessing, acting and aligning actors towards learning, it is vital that all 
appropriate inputs are supported through alignment of all relevant actors, most 
importantly the school management, community, the education department, and local 
government. 


Box 2: What about the ‘Disruptive Technologies’? 


Disruptive innovation theory, first introduced inthe mid-90s, describes innovations that 
originate in low-end footholds in existing markets (where incumbents typically focused on 


'® (Abdul-Hamid, Saraogi, & Mintz, 2017) 
'” (Andrews, Pritchett, Samji, & Woolcock, 2015) 
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their most profitable and demanding customers and paid less attention to less-demanding 
customers) or that create new-markets as footholds'®. Some of these innovations ‘sustain’ 
if they continue delivering better products and services, generally at lower costs, than 
incumbents and become mainstream. It is rare that a technology or product is /nherently 
sustaining or disruptive; disruption is a process, and smart disrupters improve their 
products and drive upmarket. 


While analyzing industries, therefore, it is pertinent to discuss ‘new technologies’, monitor 
their trajectory, and see if they emerge as successful disruptors over a period. Michael 
Porter, a Harvard Business School professor, analyzed the advent of new technologies as 
consisting of three ‘waves’!?. Wave 1 (1960s and 1970s) automated individual activities in 
the value chain, such as bill payments, maintaining records, computer-aided design etc. 


They largely improved the periphery of the education system by enhancing the productivity 


of certain activities. Wave 2 (1980s and 1990s), powered by the rise of the internet, led to 
coordination and integration across individual activities with outside suppliers, channels, 
and customers; and across geography. It allowed universities, for example, to closely 
integrate research networks. 


Wave 3 of technology being unleashed now is about technology becoming an integral part 
of the product or service itself. Embedded sensors and processors power products in the 
augmented reality and virtual reality domains; cloud computing and internet of things 
connect products in never-imagined ways; big data powers dramatic software applications 
in artificial intelligence and machine learning; and ledger technologies such as blockchain 
can fundamentally transform transactions. Very early experiments with some of these in 
the education system also exist. Augmented reality headsets have promised ‘immersive’ 
learning experiences, machine learning algorithms have been deployed to predict student 
outcomes, and blockchain-based educational certificates are already being piloted. 


Has the education industry been ‘disrupted’ in the past? Perhaps history holds a lesson. 
Johann Gutenberg's invention of the printing press inmid 1400s disrupted the access to 
information, which was earlier limited to copying and production of the written word into 
laborious manuscripts accessible to wealthy elites. Technology to create nearly identical 


'8 (Christensen, Raynor, & McDonald, 2015) 
'9 (Porter & Heppelmann, 2014) 
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books of quality at an economical price created a massive new-market foothold: the 
general public who were non-consumers of books. This disruption shifted the entire 
educational practice in the world: the relationship between the educator and student now 
had a third ingredient - technical texts as silent instructors. 


Will the onslaught of the third wave of technology disrupt education value chain (and at 
several points)? Like most cases of possible disruption, it is too early to say. Large-scale 
education has been an extremely resilient enterprise in that sense. Part of the reason is 
that it is hard to find examples of disruptive innovation in the public sector, which 
constitutes a major chunk of basic education delivery around the world. Several factors, 
such as profit motives and competitive pressures propel disruptive innovations in the 
private sector, butare largely absent in the public sector. Even when the public sector is 


innovating, the most common types of innovation often drive costs up 7°, as they tend to 


add increasingly complex and expensive features onto existing innovations (think of 
introducing smartboards in classrooms). In such ‘traditional’ innovations, price vs 
performanceis a trade-off: it is generally assumed that better performancerequires more 
teachers, smaller class sizes, and better facilities (i.e. paying higher price). Disruptive 
innovations need to break such trade-offs and reduce costs by finding footholds, i.e. a 
population of underserved/unserved consumers, and then going mainstream. ‘Online 
education’ models (such as Massive Open Online Courses, or MOOCs), might have found 
such a new-market foothold by targeting non-consumers of education. If online education 
products continue to improve and get driven upmarket, we might have a case of successful 
disruption. 


Amidst the rapid technological progress of the present millennium, one thing is certain: the 


status quo of the education enterprise might, finally, be changing. 


4. Conclusion and food-for-thought for 
policymakers 


“If all you have is a hammer, everything looks like a nail.” 


20 (Eggers, Baker, & Vaughn, 2013) 
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- Abraham Maslow 


The challenges faced in education are multi-dimensional, and vary immensely from region 
to region, many a times within the same country. Over the past two decades, anumber of 
ICT tools have been developed and implemented in the school systems around the world. 
However, technology - no matter how well designed - is only a magnifier of human intent 
and capacity; itis not a substitute". It is often tempting to think of ICTs as a silver-bullet 
solution, and jump to the debate on specific inputs (what kind of laptops should | buy?) that 
might have worked elsewhere (most likely the evidence being from highly developed 
learning environments). But acquiring the technologies themselves, no matter how hard 
and expensive, may be the easiest and cheapest element in a series of elements that 
ultimately could make these technologies sustainable or beneficial. 


Effectively integrating technology into educational systems is much more complicated. It 
involves a rigorous analysis of educational objectives and changes, a realistic 
understanding of the potential of technologies, a purposeful consideration of the pre- and 
co-requisites of effectiveness of ICTs for education, and an assessment of the prospects of 
this process within the dynamics of educational change and reform. The impact of ICTs for 
education depends to a large extent on the purpose for which ICTs are used. For example, 
if technology is used to stream recorded lectures/videos of existing teachers, and software 
is used to merely provide digital copies of existing textbooks, it is far-fetched to expect 
learning results significantly different from classroom lecturing or textbook use. However, 
these instructional technologies may extend educational opportunities to situations where 
there is no lecturer or textbook. In simple terms, understanding what one wants to 
accomplish, understanding the users and beneficiaries, and understanding the relevant 
"use cases", is the key. Development efforts utilizing ICT tools that are more likely to 
succeed are informed by co-creation and take place as close to the user as possible. 


Integrating technology into the educational processis anintricate, multifaceted process 
that involves a series of deliberate decisions, plans, and measures. As evidence” indicates, 
scaling any intervention from a ‘proof of concept’ stage requires multiple iterations of 
experimentation, in which underlying mechanisms are identified and the modelis refined 


21 (Toyoma, 2015) 
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based on this understanding. The following??*4 might serve as a step-by-step guide to 


policymakers when thinking about ICT-enabled improvement of school education systems: 


e Create a vision for ICT use in education, identifying educational areas for ICT 


intervention, formulating ICT-in-education policy, and articulating what precisely will 


the ICT intervention change/enable 


e Understand the current context, for all aspects identified in the vision for ICT use in 


education, so as to identify/build the ‘right technology’, and then develop an 


implementation strategy that helps achieve the vision, by accounting for challenges 


and leveraging strengths of the current system 


(2) 


Consult a range of actors, such as students, parents, teachers, teacher 
trainers, administrative officials, and civil society organizations 

Identify challenges and strengths related to the quality and relevance of 
content, ease and cost of set-up and maintenance of infrastructure, 
hardware and software, and knowledge, skills and mindsets of relevant 
actors 

Empower all key actors associated with the education system towards 
performing their duties and realizing their aspirations 

Train relevant actors, supportthem in deploying technology and create the 
necessary culture for the use of ICT through awareness drives across the 
system 

Proactively create adequately contextualized and tested 
content/reports/outputs that will form the heart of the intervention 

Allocate funds for maintenance and update of the infrastructure, i.e. both 
software and hardware 

Prioritize pro-equality as a fundamental tenet of the strategy 


e Monitor, evaluate, continuously learn, and improve design and implementation 


(0) 


Allocate adequate funds and resources to the creation of an MIS that serves 
as a backbone, enabling the collection, aggregation, storage, analysis and 
synthesis of data towards improvement of the design and implementation of 
technological solutions, programs and policies; evaluate for impact 


3 (Trucano, 2016) 
24 (Information for Development Program (infoDev), 2007) 
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o Build sufficiently exhaustive practices that ensure data security and pack 
appropriate privacy provisions 


All the above steps might not be relevant in every context, but it is necessary to think of a 
‘big picture’ even with smaller interventions. 


Technologies have great potential for knowledge dissemination, effective learning, and 
efficient education services. However, to truly realize this potential, the educational policies 
and strategies needto be right, ICT-in-education policies need to be well thought out, and 
the prerequisite conditions for using these technologies must be met concurrently. 
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